Skip to content

fix: validate instantiated job models with the template's extension context - #362

Merged
wyongzhi merged 2 commits into
OpenJobDescription:mainlinefrom
wyongzhi:fix/name-length-context-none
Sep 14, 2026
Merged

wyongzhi merged 2 commits into
OpenJobDescription:mainlinefrom
wyongzhi:fix/name-length-context-none

Conversation

@wyongzhi

@wyongzhi wyongzhi commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Fixes: no linked issue (reported directly; see the repro below).

What was the problem/requirement? (What/Why)

create_job re-validates the models it instantiates, but did so without a parsing context. instantiate_model built each target model with a bare target_model(**fields), so inside the models' @field_validators info.context was None. The FEATURE_BUNDLE_1 length validators introduced in #258 are written as 512 if context and "FEATURE_BUNDLE_1" in context.extensions else 128 (or else 64), so with no context they fall back to the base limits and reject values that decode_job_template had already accepted under the extension:

Field Accepted at decode (FEATURE_BUNDLE_1) Rejected at create_job
Job.name up to 512 "String should have at most 128 characters"
Environment.name up to 512 "name must be at most 64 characters long"
EmbeddedFileText.name up to 512 "name must be at most 64 characters long"
EmbeddedFileText.filename up to 256 "String must be at most 64 characters long"

Repro (before this change): decode the template below with supported_extensions=[FEATURE_BUNDLE_1], then create_job(job_template=..., job_parameter_values={}).

specificationVersion: jobtemplate-2023-09
extensions: [FEATURE_BUNDLE_1]
name: "JJJJ…(512 chars)"
steps:
  - name: s
    script:
      embeddedFiles:
        - {name: Run, type: TEXT, data: "echo hi", filename: run.sh}
      actions: {onRun: {command: echo}}

NameIdentifierLengthMixin (#318) already returns early when no context is present, which is why step names and parameter names were unaffected; the four validators above predate that pattern.

What was the solution? (How)

instantiate_model now seeds one parsing context per call from the root template's declared extensions (model.model_parsing_context_type(supported_extensions=model.extensions or [])) and threads it through the recursion; the final construction becomes target_model.model_validate(fields, context=context) instead of target_model(**fields). Public signature unchanged. The per-field TypeAdapter.validate_python calls stay context-free (nothing there reads info.context; nested instances are not re-validated). Models that do not bind model_parsing_context_type (internal test fixtures) still get context=None. The four validators in _model.py are untouched; two comments there are updated.

Why not skip the check when the context is missing: JobName limits apply "after the format string has been resolved" (2023-09 Template Schemas, §1.1.1). A short {{Param.X}} name can resolve to 129+ characters at job creation and must still be rejected without FEATURE_BUNDLE_1; the existing test_fails_to_instantiate (v0/v1) pins this and passes unchanged. openjd-rs does the same resolved-name re-check at job creation (create_job/mod.rs, create_job_rejects_interpolated_name_exceeding_128_chars). Seeding from the declared extensions matches what decode narrowed context.extensions to.

Open question: should the per-field TypeAdapter calls receive the same context? Left out since nothing reads it; happy to add for symmetry.

What is the impact of this change?

  • Templates declaring FEATURE_BUNDLE_1 can now use the raised limits through create_job; templates without it behave as before (a 129-char resolved name is still rejected).
  • Every context-gated validator on the target models now runs at instantiation with the declared extension set instead of being skipped. That set equals what decode narrowed to, so each check reproduces its decode-time result; the full suite shows no other outcome change.
  • Direct construction of models without a context is unchanged. Calling instantiate_model (exported only from openjd.model._internal) directly on a sub-model now validates with an empty extension set rather than none, so mixin-guarded name validators apply the base limit there; the sole in-tree caller passes the JobTemplate root.
  • Public API unchanged.

How was this change tested?

  • New TestCreateJobPreservesFeatureBundle1Lengths in test/openjd/model_v0/v2023_09/test_feature_bundle_1.py (30 tests, real decode -> create_job, no mocks):
    • each of the four fields at its FEATURE_BUNDLE_1 ceiling survives create_job, plus one round trip with all four at the ceiling; together with the resolved 512-char job name below these are the six tests that failed before the fix, with the messages in the table above;
    • ceiling + 1 rejected at decode under FEATURE_BUNDLE_1; base limits (128/129, 64/65) rejected at decode without the extension;
    • resolved job name via {{Param.N}}: 128 accepted / 129 rejected without the extension, 512 accepted / 513 rejected with it;
    • static type ceilings without a context (TypeAdapter(JobName), direct Environment / EmbeddedFileText construction at ceiling + 1);
    • filename path-separator rejection both under FEATURE_BUNDLE_1 decode and without a context.
  • hatch run lint clean; hatch run test: 5825 passed, 24 skipped, 3 xfailed (baseline 5795 passed; +30 new tests).
  • OpenJD conformance runner (conformance-tests/run_openjd_cli_tests.py, openjd-cli 0.7.7 installed against this checkout): 2023-09/FEATURE_BUNDLE_1/jobs/* 12 passed / 0 failed (3 Windows-only skipped), 2023-09/base/jobs/1.1.1--* 3/0, 2023-09/FEATURE_BUNDLE_1/job_templates/* 41/0, plus the three new substitution fixtures from the companion openjd-specifications PR 3/0; the invalid cases reject with "at most 128" / "at most 512".
  • The context=None branch for model classes that bind no parsing-context type is exercised by the existing test/openjd/model_v0/_internal/test_create_job.py (its BaseModelForTesting fixtures are revision-agnostic and do not bind one).

Companion PR (separate repo, not required for this one): OpenJobDescription/openjd-specifications#184 adds three conformance fixtures that cross the job-name length boundary via parameter substitution. The existing boundary fixtures are all literal names, so the suite did not cover the "after the format string has been resolved" clause.

Was this change documented?

No user-facing documentation change; this restores behaviour the schema already specifies. The two comments in _model.py that described instantiation as context-free are updated; docstrings for the changed helpers are updated in _create_job.py.

Is this a breaking change?

No. Public API and signatures are unchanged.

Does this change impact security?

No. The change adds a validation context to an in-memory model instantiation; it does not create or modify files, directories, or trust boundaries. No threat model update needed.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…ontext

create_job instantiated the target models with a bare constructor, so the
FEATURE_BUNDLE_1 length validators on Job.name, Environment.name,
EmbeddedFileText.name and EmbeddedFileText.filename ran without a parsing
context and fell back to the base limits, rejecting values that decode had
already accepted under the extension.

instantiate_model now seeds a ModelParsingContext from the root template's
declared extensions and constructs each target model with
model_validate(..., context=...). Resolved values (the job name) are still
checked against the extension-aware limit, matching the schema text and the
Rust implementation; literal fields keep the result decode already reached.
Models whose class binds no parsing-context type are constructed without a
context, exactly as before.

The two comments in _model.py that described instantiation as running
without a context are updated to match; no validator logic changes.

Signed-off-by: Yongzhi Wei <276409147+wyongzhi@users.noreply.github.com>
@wyongzhi
wyongzhi merged commit ebac419 into OpenJobDescription:mainline Sep 14, 2026
31 checks passed
@wyongzhi
wyongzhi deleted the fix/name-length-context-none branch September 14, 2026 17:49
leongdl added a commit that referenced this pull request Sep 17, 2026
fix: apply the template's extensions in the job parameter merge

preprocess_job_parameters re-validates each merged job parameter definition
through parse_model, and it passed no supported_extensions. The parsing context
that reached NameIdentifierLengthMixin therefore carried an empty extension set,
so the mixin enforced the base 64-character limit on a parameter name that decode
had already accepted at up to 512 under FEATURE_BUNDLE_1. create_job calls
preprocess_job_parameters itself, so both entry points rejected the name.

Measured before the change, for a template declaring FEATURE_BUNDLE_1 with a
512-character STRING parameter name:

    decode_job_template      OK
    preprocess_job_parameters DecodeValidationError: 1 validation errors for
                              JobStringParameterDefinition name: name must be
                              at most 64 characters long
    create_job                same

merge_job_parameter_definitions now derives the union of the extensions declared
by the job template and any environment templates, and threads it to
merge_job_parameter_definitions_for_one and on into parse_model. The union is
sound because a definition was decoded under its own template's extension set and
every definition of one job parameter carries the same name, so the union cannot
admit a value decode did not already accept for the template that declared it.

The public signature of merge_job_parameter_definitions is unchanged: it already
receives the templates that carry the extensions, so the set is derived rather
than passed. This follows the shape of #362, which fixed the sibling defect in
instantiate_model, and it keeps a caller from declaring an extension the template
did not. merge_job_parameter_definitions_for_one takes a new keyword-only
supported_extensions argument that defaults to None, preserving its behaviour for
any existing caller.

Tests cover all four FEATURE_BUNDLE_1 length ceilings at job-creation time on
both lanes. v0 gains group H in test_feature_bundle_1.py: the parameter name
through the merge, preprocess_job_parameters and create_job for each of STRING,
PATH, INT and FLOAT, a genuine two-source merge, the environment-template-only
case, the EXPR-typed model_copy branch, and controls for the base 64 limit and
the static 512 identifier ceiling. v1 gains an equivalent class in
model_v1/test_create_job.py, which had no coverage of these ceilings at all; the
Rust implementation validates in a single pass and already passes, so those tests
are a ratchet against a re-validation pass being added upstream unnoticed.

Every v0 test was mutation-checked. Seven mutants, each caught: dropping the
threading at any of its three points, hardcoding FEATURE_BUNDLE_1 instead of
deriving it, ignoring environment templates, applying the threading only to
STRING, and applying it only to single-source merges.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants